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I have over the past several 
years been involved with Hawaii’s 
Kamehameha Schools on a project 
called the Hawaiian Cultural 
Influences in Education (HCIE) 
study. Kamehameha Schools was 
established to provide educational 
opportunities to improve the 
capability and well being of people 
of Hawaiian ancestry. 

I am using this letter to describe 
some of the more interesting 
findings from this line of work 
so far. Central to this work is the 
evaluation of effects on student 
achievement attributable to 
exposure to a Culture-Based 
Education (CBE) curriculum. I’ll 
describe CBE more completely in a 
subsequent section. 


Two key questions guide the 
development of the HCIE effort: 

1. How do culture-based educational 
strategies affect the achievement 
and growth of students? 

2. What factors mediate the 
relationship between culture- 
based education and student 
achievement/behavior (e.g., self- 
esteem, student engagement, 
cultural identity, family/community 
connections)? 

We use a wide range of survey data 
collected from teachers, students, 
and parents to examine the questions 
above. We also use data from 
administrative records on students 
enrolled in the schools participating 
in the study. We focus specifically on 
student performance in math and 
reading as our outcomes. 



Culture-based educational (CBE) 
strategies are viewed as a promising 
approach to address disparities 
between indigenous students and their 
peers (Ledward &Takayama, 2008). CBE 
represents an indigenous conceptual 
framework defined by five primary 
domains. These domains emphasize 1) 
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a focus on culturally-relevant content, 2) 
culturally relevant contexts, 3) the use 
of heritage language, 4) the involvement 
of family and community, and 5) 
indigenous forms of assessment. 

My colleague Ron Heck (University of 
Hawai'i at Manoa)and I are examining the 
impact of CBE strategies in a multilevel 
educational context where students are 
nested in classrooms and schools, each 
with its own environmental features that 
shape teacher and student attitudes and 
behaviors. 

Before I get into the nested nature 
of relationships presumed to affect 
learning outcomes (these are expressed 
in our guiding question 2), let me 
introduce the more basic correlational 
relationship expressed in guiding 
question 1 that underlies this work. 

The central idea in this question 
is captured with the widely used 
conceptual model in Figure 1 below. 
Basically, this figure suggests that CBE 
has a direct effect on learning outcomes 
(as expressed through academic 
performance) and an indirect effect on 
these outcomes that operates through 
enhanced socio-emotional development. 



Figure 1. Relating CBE to Learning 

Our guiding question 2 is more 
complex and is illustrated in Figure 2. It 
assumes a relationship between student 
achievement/behavior (as expressed 
in Figure 1 above) and focuses on why 
that relationship may be stronger or 
weaker for some students relative to 
others. Any number of factors could 
moderate this relationship. They might 
be student-level factors such as ethnic 
background or socioeconomic status, 
teacher characteristics such as time 
in the classroom, gender, or ethnicity, 
or school-level factors such as school 
size, average teacher “quality,” or the 


demographic composition of the school. 
There is also a parent/family component 
that might play a role in this relationship. 

Figure 2: Proposed Multilevel Model 
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The figure implies several 
relationships that we test using data 
from a sample of middle and high 
schools in Hawaii. Our effort here 
capitalizes on a body of methodological 
work that Ron and I have been deeply 
involved with over the past decade 
(Heck & Thomas, 2000; Heck, Thomas, & 
Tabata, 2010; Heck, Thomas, & Tabata, 
forthcoming). This methodological 
work is focused on better capturing the 
interactions between individuals and 
the contexts in which they are nested. 
That is, we are interested in examining 
how contexts can shape behaviors and 
learning outcomes. We use a range of 
multilevel models to operationalize 
the conceptual model in Figure 2. 

How do we measure these things? 

We include a CBE teaching rubric on 
the teacher survey. This rubric measures 
critical indicators in the five domains 
with Likert-type scales. Each domain is 
described in behavioral terms to indicate 
different levels of implementation 
(Ledward & Takayama, 2008). Our 
statistical models suggest that CBE as a 
construct is well measured by the items 
included on the teacher survey. 

We measure student affect using 
items operationalizing the socio- 
emotional development construct in 
Figure 1. These items are incorporated 
on the student survey and include 
questions about self-efficacy, cultural 
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affiliation, cultural knowledge and 
practice, connection to ‘ohana (family), 
and connection to the community. As 
with the CBE construct, we find that the 
socio-emotional (SE) construct is also 
well measured by the range of variables 
included on the student survey. We 
were also able to establish that student 
socio-emotional development affects 
achievement outcomes that we are 
interested in. Thus our results are 
consistent with and supportive of the 
model depicted in Figure 1. 

A three-level model 

While I’ll avoid the details of the 
statistical models here, I’ll state simply 
that our results are based on a series 
of three-level analyses of achievement 
using data from 1,055 students cross- 
classified with 245 teachers, in 43 Hawai’i 
schools. This approach allows us to 
decompose the variance in achievement 
across the three levels and to associate 
those variances with explanatory factors 
at each level (e.g., teachers, students, 
or schools). Moreover, we can also test 
for effects resulting from interactions 
across the levels. For example, we might 
want to estimate the effect of the school 
environment on teachers’ impact on 
student development and achievement. 
This is a relationship implicit in Figure 2. 

What are the major findings? 

Finding #1: Teacher implementation 
of CBE is contingent on “average” 
school CBE implementation. Our 
results show us that, in math, teacher 
implementation of CBE is contingent on 
the average level of CBE implementation 
in the school; this supports the type of 
“cross-classified” relationship proposed 
in Figure 2 (i.e., see the vertical arrow 
between schools and classrooms). More 
specifically, we find that the effect of 
individual teacher use of CBE principles 
at the classroom level is moderated by 
the average school implementation of 
CBE. In reading, average teacher CBE 
implementation is also significantly 
related to reading levels. Moreover, 
this effect is once again contingent on 
the average implementation of CBE at 
the school level. In other words, the 
findings support the notion that school 
implementation of CBE forms a context 
for this type of instruction that positively 
affects student learning. Through our 


analysis we find that in reading, the 
percentage of fully qualified teachers 
in the school also interacts with teacher 
implementation of CBE in the classroom. 
Our results provide support for the idea 
that individual teacher commitment 
to CBE matters (that is, it is positively 
associated with student performance 
in reading), but that the broader school 
environment powerfully moderates their 
impact on student performance. In math, 
we note teacher implementation of CBE at 
the classroom level enhanced the effect 
of student socio-emotional development 
on achievement. This provides evidence 
of another cross-level effect proposed 
in Figure 2. In follow-up analyses we 
also find that teachers who are “high 
implementers” of CBE have a particularly 
strong positive (and significant) effect on 
student learning in math. 

Finding #2: Parent perception of 
student socio-emotional growth is an 
important predictor of student and 
school achievement. In a separate 
analysis we focus on parent perceptions 
measured at the individual student 
and school levels and their impact on 
student learning. Our results suggest 
a positive relationship at both levels, 
controlling for school composition (and 
average teacher quality). 

We next add parent perceptions 
to CBE implementation. The analyses 
adding the parent variables are based on 
a subset of the data. 

Our results are consistent with the 
idea that parent support at the school 
level is an important predictor of 
student achievement outcomes. Our 
findings suggest the positive impact of 
parent perceptions of their children’s 
socio-emotional development on 
achievement at the school level. Once 
again, we find that CBE impact in the 
classroom on achievement in reading 
and math is contingent on average CBE 
implementation at the school level. In 
this area we also find that individual 
parent perceptions of their child’s 
socio-emotional development enhanced 
achievement in math (but not reading). 

Finding #3: CBE enhanced the effect 
of student socio-emotional development 
on math achievement. Our analysis also 
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suggests that teacher implementation 
to CBE principles enhanced the effect of 
student socio-emotional development 
on math achievement. This finding does 
not hold in reading, however. 

Discussion 

Overall the results provide 

considerable support for the proposed 

three-level model of individual, 

classroom, and school influences 

on student learning. The models 
take advantage of considerable 
student, teacher, and school data. 
First, at the student level, the results 
confirm that aspects of students’ 
backgrounds exercise some influence 
on their achievement (e.g., gender, 
socioeconomic status, race/ethnicity). 
These factors have been well established 
in previous research. Similarly, previous 
achievement (e.g., in either reading or 
math), as we might expect, contributes 
to later levels of achievement in both of 
those areas. 

In math, the student affect variable 
is positively related to outcomes. 
Recall that this is a composite indicator 
comprised of variables tapping self- 
efficacy, cultural affiliation, cultural 
knowledge & practice, connection to 
‘ohana, and connection to community. 
Also, higher levels of teacher CBE 
implementation increased the 
relationship between student affect and 
achievement in math. 

In reading, however, we could find 
no relationship. We think this may be 
an anomaly as our more detailed follow- 
up analyses with students from a subset 
of schools with high proportions of 
students of Native Hawaiian ancestry 
showed that student affect is significant 
and positive in explaining both reading 
and math outcomes. We also find that 
parent support is positively related to 
both student achievement outcomes. 
This provides support for the idea that 
the parent variables (at the individual- 
student level) contribute meaningfully 
to explaining individual-student 
outcomes. 

Second, at the classroom/ 
teacher level, the focus is primarily 
on teacher implementation of CBE. 
The findings add something new in 
terms of demonstrating the added 


value of school CBE in moderating 
the relationship between individual 
teachers’ implementation of CBE and 
student achievement. We noted that 
the individual teacher effect in this 
area seems to be related to the school’s 
average implementation of CBE. In both 
the reading and math models, stronger 
CBE average implementation at the 
school level is related to a stronger effect 
of individual teacher implementation. 

This provides support for a key cross- 
level relationship (i.e., a vertical arrow in 
Figure 2) implied within the multilevel 
conceptual framework. Central to a 
multilevel framework is the fact that 
conditions at higher levels of the data 
hierarchy are assumed to directly 
influence relationships at lower levels. 
These cross-level interactions provide 
important means through which strategic 
actions at the school level can have 
“coordinating” effects on relationships 
at classroom levels. In a number of 
preliminary (e.g., two-level) analyses, we 
also find that teacher demographics (e.g., 
gender, education, experience, years in 
Hawai'i, Hawaiian studies major, Hawaiian 
language course) have no direct impact 
on student learning at the classroom 
level. The aggregated parent variable (as 
a classroom composition indicator) also is 
not directly related to student achievement 
at the classroom level. 

Our intermediate analyses of the 
composites also reveal that each 

separate indicator of CBE is significantly 
related to the student affect composite. 
This suggests each of the indicators 
is useful in defining CBE. The student 
affect indicators, however, work best 

as a composite. In our intermediate 

analyses of that measure, we find self- 
efficacy and ‘ohana alone to be related 
to the composite CBE measure; however, 
the other indicators separately are not 
related to teacher CBE. 

Third, at the school level, two 

variables have consistent influence on 
differences in school outcomes. The 
first is the percentage of fully qualified 
teachers in each school. Schools having 
higher percentages of fully qualified 
teachers have higher outcomes in both 
reading and math. The second is average 
parent support (aggregated from the 


student level), which is positively related 
to outcomes in reading and math at 
the school level. This provides further 
evidence of the importance of this 
variable, in that it works as a type of 
composition effect (i.e., it demonstrates 
that effects of parent support tend to 
accumulate across levels of the data 
hierarchy). 

Taken together, the results of our 
multilevel analyses suggest that CBE is 
an important predictor of achievement, 
contingent on the school’s overall level 
of implementation of these principles. 
We note that the three major constructs 
at the center of this research (i.e., teacher 
CBE, student affect, and achievement) 
seem to work in expected ways. More 
specifically, our results provide evidence 
that teacher CBE (at either the school or 
teacher level) is related to both student 
affect and achievement. This lends 
validity to the instruments through which 
they were defined and measured, as 
well as their observed effects within our 
proposed conceptual model to explain 
student learning outcomes. Similarly, 
in most instances, the student affect 
measures we used are positively related 
to achievement— more definitively 

in math, but also in some subsets of 
the data on reading outcomes. These 
findings are quite consistent across the 
various models we have specified. 

Limitations 

Two primary limitations should be 
noted in considering the results of our 
analyses at this point. First, questions 
remain about the definition of teacher 
CBE implementation in the classroom. 
Differences in preparation, instructional 
strategies, learning activities, and social 
contexts are likely embedded in the 
concept of effective implementation 
and delivery of principles contained 
in the CBE construct. It is also unclear 
from the present data what the actual 
mechanisms are that link differences in 
the quality of schools’ implementation 
of CBE, the quality and stability of 
their teaching staffs, and the collective 
effectiveness of CBE implementation on 
the implementation of CBE by individual 
teachers (especially how students’ 
exposure to different combinations 
of teachers may influence affect and 
academic performance). As a classroom- 
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level measure, teacher implementation 
of CBE (as a set of survey items) leaves 
unanswered questions about how the 
effectiveness of CBE implementation 
might result from differences in teachers’ 
expectations, curriculum coverage, 
and domain-specific learning activities 
(as well as how school implementation 
might contribute). Similarly, as a 
school-level variable, collective teacher 
CBE implementation is only a proxy 
for more thorough information that 
could be aggregated about teachers’ 
expertise and professional learning. 
Although upgrading teacher CBE skills 
represents a promising avenue for 
school improvement, we still need to 
identify key pedagogical strategies 
and behaviors in each subject domain 
that enable more effective teachers (in 
terms of CBE usage) to foster students’ 
academic growth in diverse school 
settings. This might be enhanced by up- 
close classroom measures (e.g., logs, 
observations) of teachers’ teaching 
plans, assessments of their lesson 
performance, assignments and feedback 
for students, and so forth. 

A second limitation is embedded 
in the rudimentary way we begin to 
address questions about temporal 
sequences underlying associations 
between teachers and student outcomes. 
Although our analyses include a measure 
of student achievement at time 1, the 
measures regarding implementation 
of CBE are only collected on one 
occasion. This has been a limitation 
of previous cross-sectional research 
on teacher effectiveness (Goldhaber, 
2002; McCaffrey et al., 2003; Seidel 
& Shavelson, 2007). This approach is 
limited in that it begs the question, 
“do the observed results represent a 
difference in the quality of teachers’ skills 
and implementation or some type of a 
selection effect?” It is possible, however, 
that over time more highly-qualified 
(and effective) teachers seek schools 
with particular student composition 
or where educational processes and 
student outcomes are stronger. 

Despite these two limitations, the 
results offer some promising avenues 
for policy development and further 
research. The results and limitations of 
this study suggest that work on teacher 



implementation of CBE should proceed 
in at least 3 directions. 

1. There is a need to provide clear linkages 
between teachers’ professional 
preparation, qualification, and 
professional development in terms of 
implementing CBE. This in turn needs 
to be connected to their expertise in 
domain-specific teaching activities, 
and classroom effectiveness. This will 
likely occur with increased facility to 
link students and their teachers within 
educational accountability systems. 

2. The accurate and valid measurement 
of how the differential effectiveness 
of several teachers in terms of CBE 
skills contributes to students’ learning 
growth across a variety of classroom 
contexts remains a fruitful area of 
inquiry. This is especially important 
in policy debates about the nature 
and features of evaluation systems 


that hold teachers accountable for 
students’ progress. Future research 
might examine whether school 
conditions moderate the size of 
classroom/teacher effects described 
in our work. This can be accomplished 
through larger unit samples that will 
allow more complete specification of 
the conceptual model. 

3. Finally, combining detailed information 
about what teachers do in classrooms 
to facilitate learning and what school 
leaders can do to build the school’s 
professional capacity should contribute 
to a more complete understanding 
about how teacher skills and 
effectiveness, as school resources, 
affect student learning under a variety 
of classroom and school conditions. 

Our work on these fronts continues 
and I will report on our progress in a 
future letter. 
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Multilevel and Longitudinal Modeling with IBM SPSS 


This is the first book to demonstrate how to use the multilevel and 
longitudinal modeling techniques available in IBM SPSS Version 18. The 
authors tap the power of SPSS’s Mixed Models routine to provide an 
elegant and accessible approach to these models. Annotated screen shots 
with key output provide a step-by-step understanding of each technique. 
Diagnostic tools, data management issues, and related graphics are 
introduced throughout. SPSS commands show the flow of the menu 
structure and how to facilitate model building. Annotated syntax is also 
available for those who prefer this approach. Most chapters feature 
an extended example illustrating the logic of model development that 
demonstrates the context and rationale of the research questions and the 
steps around which the analyses are structured. 

The book opens with the conceptual and methodological issues associated 
with multilevel and longitudinal modeling, followed by adiscussion of SPSS 
data management techniques which facilitate working with multilevel, 
longitudinal, and/or cross-classified data sets. The next few chapters 
introduce the basics of multilevel modeling, how to develop a multilevel 
model, and trouble-shooting techniques for common programming and 
modeling problems along with potential solutions. Models for investigating 
individual and organizational change are developed in chapters 5 and 6, 
followed by models with multivariate outcomes in chapter 7. Chapter 8 
illustrates SPSS’s facility for examining models with cross-classified data 
structures. The book concludes with ideas on expanding the various 
multilevel and longitudinal modeling techniques introduced and issues to 
keep in mind in conducting multilevel analyses. Ideal as a supplementary 
text for graduate level courses on multilevel, longitudinal, latent variable 
modeling, multivariate statistics, and/or advanced quantitative techniques 
taught in departments of psychology, business, education, health, and 
sociology, this book’s practical approach will also appeal to researchers 
in these fields. The book provides an excellent supplement to Heck & 
Thomas’s An Introduction to Multilevel Modeling Techniques, 2nd Edition; 
however, it can also be used with any multilevel and/or longitudinal 
modeling book or as a stand-alone text. 
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Thomas’ substantive work bridges K-12 and higher education and focuses on issues related to the stratification of educational 
experiences and opportunities. Topics revolving around college access and success are central themes in his research and 
teaching. This work can be found in a range of refereed journals, monographs, reports, and book chapters. 

In addition to his substantive work, Thomas has devoted a considerable amount of effort to research on methodological issues. 
As much of his substantive work concerns the structural influences on individual behavior and opportunity, the development and 
explication of statistical methods capable of modeling such phenomena have also attracted his interest. His work in this area can 
be found in books, An Introduction to Multilevel Modeling (with Ron Heck) or Multilevel and Longitudinal Modeling with PASW 
(SPSS) (with Ron Heck and Lynn Tabata), and related articles in a number of refereed journals and book chapters. 

Thomas is the editor of the Journal of Higher Education, one of the field’s top journals. He has served as guest editor for the 
American Education Research Journal and is a co-editor of the International Studies in Higher Education series. He is a regular 
reviewer for several discipline specific journals and currently serves on the editorial board of the journal Research in Higher 
Education, the annual Higher Education: Handbook of Theory and Research, and on the Reading on Equal Education advisory 
panel. 

He teaches a range of substantive courses in education including Federal Higher Education Policy, State Higher Education Policy, 
Stratification in Higher Education, and The University & Society. 
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